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INTRODUCTION 


SSS  is  pleased  to  present  this  Final  Report,  recounting  activities 
performed  as  part  of  the  project  entitled:    "Enhancement,  Validation  and 
Analysis  of  Central  Office  Statistical  Files"  (Contract  Number  500-86-0021) . 
This  report  is  a  high  level  summary  of  activities  which  we  completed  in 
accomplishing  the  required  tasks.    More  detailed  reports  have  previously  been 
submitted,  covering  the  following  areas: 


o  the  November  1986  BMAD  Conference; 

o  our  review  of  1984  BMAD  Data; 

o  our  review  of  1985  BMAD  Data;  and 

o  the  completion  of  Tasks  2  and  3. 

The  reader  is  referred  to  those  reports  for  a  more  detailed  discussion  of  both 
the  activities  completed  and  results/recommendations. 

This  report  is  divided  into  six  sections,  which  correspond  to  the 
five  tasks  specified  in  the  Statement  of  Work  and  a  final  section  which 
includes  our  conclusions  and  recommendations. 
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TASK  1  -    Orientation  and  Conferences 


Objective:       To  organize  a  series  of  training  sessions  for  carrier  personnel 
responsible  for  the  preparation  of  BMAD,  in  order  to  increase  their 
understanding  of  the  CY1985  (and  subsequently  CY1986)  BMAD  instructions. 


Activities:     The  following  activities  were  completed  as  part  of  this  task: 

o      Assisted  HCFA  staff  in  organization  and  chairing  of 

meetings  of  the  BMAD  Conference  Steering  Committee;  the 
Committee  met  approximately  twice  a  month  from 
September  1986  until  September  1987;  Committee  members 
had  significant  input  into  the  CY1986  and  CY1987  BMAD 
instructions  as  well  as  the  content  of  the  Conference 
agenda  and  materials. 

o      Prepared  all  materials  for  Conference  Steering 
Committee  meetings. 

o      Completed  conference  site  negotiations  and 
arrangements; 

o      Completed  conference  pre-registration  activities; 
for  the  first  conference  this  included  a 
significant  amount  of  coordination  with  BDMS,  BPO, 
and  Regional  Offices  to  ensure  that  appropriate 
carrier  personnel  attended; 

o      Assisted  BDMS  in  development  of  all  conference 
teaching  materials,  including  several 
questionnaires; 

o      Produced  and  distributed  all  conference  materials; 

o      Served  as  moderator  for  conference  speakers,  and 
presented  edit  results  in  second  conference; 

o      Provided  on-site  conference  support; 

o      Provided  post-conference  support,  including 
processing  of  reimbursement  requests; 

o      Provided  post-conference  summary  reports  to 

members  of  the  Conference  Steering  Committee;  and 

o      Provided  recommendations  to  BDMS  on  content  and 
format  of  CY86  and  CY87  BMAD  instructions. 
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TASKS  2  and  3    -     Development  of  Recommendations  for  Enhancement  of  Provider 

and  Beneficiary  Files 

Objective:  The  purpose  of  these  tasks  was  to  enhance  the  1984  BMAD  Provider 
and  Beneficiary  files  for  selected  carriers,  with  certain  claims 
history/fee  data.  Specifically,  SSS  was  to  make  recommendations 
to  HCFA  related  to  adding  customary  charge  data  to  the  Provider 
file  (BMAD  III)  and  adding  locality  data  to  the  Beneficiary  file 
(BMAD  IV).  (Beginning  with  1985  data,  locality  is  now  a  required 
field  on  the  BMAD  IV  file.) 


Activities:     Due  to  the  similarity  of  the  two  tasks,  most  activities  were 
carried  on  concurrently  for  both  tasks,  as  described  below: 

o      Reviewed  all  available  documentation  on  the  BMAD 
system  and  carrier  claims  processing  systems; 

o       Interviewed  a  number  of  HCFA  central  office  staff 
to  obtain  information  on  the  BMAD  system, 
including  representatives  from  BPO,  BDMS,  and 
BERC; 

o      Developed  detailed  work  plan  for  completion  of 
Tasks  2  and  3,  including  preliminary  selection  of 
appropriate  HCPCS  procedure  codes  for  customary 
charge  enhancement  (it  was  subsequently  determined 
that  it  would  be  more  efficient  to  obtain  all 
customary  charges  available  rather  than  excluding 
certain  procedures); 

o      Contacted  carrier  personnel  and  obtained  required 
data  and  documentation  for  enhancement; 

o      Based  upon  available  data  and  documentation, 

developed  workplan  to  obtain  enhancement  data;  due 
to  significant  variations  in  carrier  procedures,  a 
unique  approach  was  necessary  for  each  carrier, 
which  are  described  in  more  detailed  in  our 
previously  submitted  report  on  these  two  tasks;  in 
general,  due  to  the  non-specific  provider  ID 
number  used  in  the  BMAD  Provider  and  Beneficiary 
files,  the  Carrier  Control  Number  was  used  as  the 
link  between  the  BMAD  file  and  the  Claims  History 
file  in  order  to  obtain  a  specific  provider  ID 
number  and  locality;  the  customary  charge  was  then 
obtained  from  a  separate  customary  charge  file; 

o      Completed  various  analyses  and  sample  merges  of 
the  files  in  order  to  develop  recommendations  for 
completing  the  proposed  enhancement;  in  those 
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cases  where  adequate  data  was  available,  completed 
actual  enhancement;  and 

o      Completed  final  report,  including  recommendations  for 
enhancement;  differences  among  carriers  and  lack  of 
adequate  documentation  required  extensive  consultation 
with  carrier  personnel  to  effectively  review  data  and 
develop  appropriate  recommendations. 


TASK  4  -    File  Validation  and  Cleansing 

Objective:       The  purpose  of  this  task  was  to  develop  recommendations  for 
correcting  invalid  or  incomplete  CY1984  BMAD  data,  and  to 
recommend  a  process  for  validation  and  cleansing  of  CY1985  BMAD 
data. 


Activities:     The  following  activities  were  completed  as  part  of  this  Task: 

o      Reviewed  available  documentation  on  the  BMAD 

system  and  various  carrier  processing  systems  and 
interviewed  HCFA  personnel  as  described 
previously; 

o      Completed  site  visits  to  two  carriers; 

o      Made  copies  of  available  1984  files  for  six 
carriers; 

o      Reviewed  frequencies,  cross-tabulations  and  record 
dumps  for  available  1984  files; 

o      Developed  system  of  edits  and  analytical  reports 
and  ran  on  available  1984  files; 

o      Wrote  Working  Paper  and  Final  Report  on  1984  files 
reviewed;  presented  results  of  review  to  BDMS  and 
ORD  representatives; 

o      Revised  edits  and  used  in  review  of  1985  BMAD 
files;  assisted  BDMS  in  analysis  of  results  and 
made  presentation  of  results  at  second  BMAD 
conference  (June,  1987);  and 

o      Wrote  Final  Report  on  1985  BMAD  data,  which 

includes  evaluation  of  quality  of  the  1985  data, 
and  recommendations  for  validation  of  future  BMAD 
submissions. 
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TASK  5    -  Statistics  and  Statistical  Tests 


Objective:       The  purpose  of  this  task  was  to  provide  statistical  data 

processing  support  services  for  completion  of  various  analyses  by 
ORD  researchers. 


Activities:     The  following  activities  were  completed  as  part  of  this  Task: 

o      Copied  1982  and  1984  Long-Term  Care  Survey  (LTC) 
files  and  converted  to  SAS  format; 

o      Completed  various  analytical  programming  tasks  on 
the  1982  and  1984  LTC  files,  including:  analysis 
of  original  variables  (frequencies,  cross- 
tabulations,  etc.);  construction  and  analysis  of 
new  variables;  creation  of  specified  analysis 
files;  comparison  of  1982  and  1984  LTC  data; 
merging  of  Area  Resource  File  (ARF)  data  and 
creation/  analysis  of  new  LTC/ARF  variables; 

o      Prepared  numerous  tables  and  reports  for  ORD 

researchers  utilizing  the  1985  BMAD  master  files, 
including:    completion  of  ranking  by  frequency  and 
allowed  charges  for  selected  HCPCS  codes  in 
various  groupings;  production  of  cross-tabulations 
for  various  Type  of  Service/HCPCS  groups; 
development  of  format  and  1985  version  of  the 
"BMAD  Data  Book",  which  will  be  an  annual 
compilation  of  summary  statistics  from  the  BMAD 
Master  files; 

o      Completed  HCPCS  coding  as  part  of  analysis  of 

frequency  of  BC/BS  selected  laboratory  procedures; 
and 

o      Provided  analytical  support  to  researchers  in 
developing  specifications  for  creation  of  tables 
using  BMAD  data,  to  ensure  that  specifications 
were  consistent  with  various  characteristics  of 
the  data. 
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CONCLUSION 


As  stated,  we  have  provided  detailed  conclusions  and  recommendations 
on  the  1984  and  1985  BMAD  data  in  previously  submitted  reports.    The  following 
are  some  general  conclusions  and  corresponding  recommendations  related  to  the 
BMAD  file  development  process,  which  have  resulted  from  our  involvement  with 
this  process  since  September,  1986. 


1.  The  BMAD  instructions  development  process  should  be  streamlined. 

Our  participation  on  the  committee  which  performed  the  in-depth 
review  of  the  annual  instructions  for  1985  and  1986  brought  out  several  points: 

o     If  the  review  group  cannot  (quickly)  agree  on  the  precise 
meaning  of  a  variable  (field),  there  is  little  or  no 
likelihood  that  the  contents  of  the  field  in  question,  as 
provided  by  the  various  carriers,  will  be  useful.  Perhaps 
even  worse,  users  of  that  field  may  subsequently  waste 
considerable  time  before  coming  to  the  same  conclusion,  and 
BDMS  personnel  may  spend  inordinate  amounts  of  time 
discussing  such  variables  with  the  carrier's  programming 
staff. 

o     Some  fields  would  more  appropriately  be  classified  as 

carrier-level  than  record-level  data  because,  for  a  specific 
carrier,  the  value  is  the  same  for  every  record. 

o     There  is  a  tendency  in  the  process  to  always  add  more 

variables  and  never  eliminate  any.    Even  outdated  fields,  or 
those  which  contain  ambiguous  data,  are  frequently  not 
eliminated.    Decisions  in  this  area  would  be  easier  if  the 
review  process  Quick! v  led  to  printed  instructions,  which 
were  readily  comparable  with  past  instructions  and  which 
could  be  circulated  to  other  researchers  for  comments  (see 
recommendation  2) . 


2.  Additional  production  support  is  needed  in  order  to  efficiently 

produce  annual  BMAD  instructions,  reflecting  all  required  revisions. 

Annual  instructions  containing  specifications  for  BMAD  files  must  be 
produced  for  the  carriers.    Because  of  the  number  of  different  sources  of  input 
to  the  document  and  the  number  of  required  HCFA  review  levels,  this  has  been  a 
complex  process.    In  addition,  the  slow  turn-around  of  word  processing  services 
related  to  this  activity  has  considerably  restricted  the  frequency  and  volume 
of  revisions.    This  delay  in  production  is  undoubtedly  due  in  part  to  the 
current  instruction  format.    Even  insignificant  changes  impact  pagination  of  at 
least  one  of  the  four  sections  of  the  instructions.    This  requires  that  the 
entire  document  be  reprinted,  and  it  becomes  difficult  to  compare  "before  and 
after"  versions  to  verify  that  only  desired  changes  have  been  implemented. 
Clearly  the  instructions  for  each  field  should  begin  on  a  separate  page.  Also, 
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the  current  multiple  column  format  needlessly  complicates  word  processing  and 
this  feature  is  not  necessary. 

In  addition,  there  appears  to  be  a  need  for  provision  of  production 
support  services  for  the  BMAD  instructions,  including  word  processing,  editing, 
duplication,  and  distribution  tasks  required  in  the  instructions  development 
process.    There  should  be  a  central  repository  for  revisions  coming  from 
numerous  sources,  thereby  ensuring  that  the  document  is  of  high  quality  and 
consistency.    It  is  extremely  important  that  these  services  provide  rapid 
turnaround,  to  allow  substantially  more  time  for  author  review  and  revision 
than  is  currently  available. 


3.  An  analysis  of  the  effect  of  inconsistent  reporting  of  adjustment 

data  in  the  BMAD  files  is  required. 

Currently,  claims  data  which  includes  subsequent  adjustments  to 
original  payment  amounts  are  reported  inconsistently  in  the  BMAD  files.  Many 
carriers  are  reporting  the  data  inaccurately,  citing  the  significant  processing 
expense  required  to  report  accurate  final  payment  amounts  (i.e.,  reflecting  an 
unduplicated  count  of  original  payments  and  all  subsequent  adjustments).  A 
major  unresolved  issue  is  the  volume  of  services  and  associated  charges 
represented  by  such  data.    Upon  resolution  of  that  issue,  HCFA  can  then 
determine  the  cost/benefit  of  mandating  accurate  reporting  of  adjustment  data. 
We  strongly  recommend  that  this  issue  be  resolved  prior  to  the  carrier-wide 
conversion  to  the  CABLE  system  since  at  least  one  of  the  carrier  test  sites 
reportedly  can  not  provide  adjustment  data  as  planned  in  the  CABLE  system.  In 
this  regard,  we  recommend  that  the  following  types  of  analyses  be  completed: 

o     An  independent  analysis  of  selected  carrier  history 
files,  to  determine  effect  of  adjustments  on  1985,  1986 
BMAD  data. 

o     An  analysis  of  selected  carriers'  procedures  for 
processing  adjustments  in  development  of  customary 
charge  profiles,  in  order  to  identify  best  approach  for 
reporting  of  adjustment  data. 

Until  an  estimate  of  the  volume  and  charges  associated  with  adjustments  is 
obtained,  the  unknown  effect  of  these  transactions  will  represent  a  significant 
unresolved  issue  for  users  of  BMAD  data. 


4.  A  comprehensive,  automated  quality  control  system  is  required  in 

order  for  BMAD  data  to  be  validated  in  a  timely  manner. 

Due  to  the  limited  number  of  BDMS  edits  used  on  the  1985  data,  and 
the  lack  of  automated  analytical  reporting,  the  quality  control  measures  used 
to  determine  acceptance  of  the  files  were  necessarily  limited  and  were  a  con- 
tributing factor  in  the  problems  with  the  1985  data.    We  recommend  that  an 
automated  review  process,  such  as  that  which  we  used  in  the  completion  of  Task 
4,  be  implemented  by  BDMS.    A  system  of  this  type  is  needed  due  to  the  size  of 
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the  BMAD  files  and  the  existence  of  problems  unique  to  various  carriers.  This 
review  system  would  provide  not  only  detailed  tabulations  and  exception  re- 
ports, but  also  generate  sufficient  machine-readable  detail  to  permit  com- 
parisons across  carriers,  by  years,  and  region.    Within  a  very  brief  time  after 
receipt  of  BMAD  data  by  HCFA,  these  general  reports  should  be  available.  This 
would  allow  BDMS  analysts  to  track  anomalies  in  both  directions:    from  the 
general  to  the  specific  and  vice  versa. 

The  core  of  the  proposed  review  system  is  a  series  of  "critical"  and 
"non-critical"  edits,  as  recommended  in  our  Final  Report  on  1985  data.  (Many 
of  these  edits  have  been  implemented  by  BDMS  for  review  of  1986  data.) 
Critical  edits  are  those  which  define  minimal  quality  standards,  and  should 
have  a  maximum  tolerance  level.    Edits  classified  as  non-critical  can  be  used 
for  two  purposes:    1)  as  an  indicator  of  possible  erroneous  data  when  applied 
to  longitudinal  data  (i.e.,  although  not  indicative  of  an  error  across 
carriers,  they  may  flag  erroneous  data  when  results  are  not  consistent  for  a 
given  carrier  across  years);  and  2)  as  an  indicator  of  carrier  specific 
aberrations  which  may  be  of  interest  to  a  researcher  using  the  file. 

The  other  necessary  component  of  the  BMAD  quality  control  system  is  a 
series  of  analytical  reports  to  complete  various  types  of  analyses  of  the  BMAD 
data:    1)  within  a  file  category  ( I - I V)  -  comparison  of  current  data  across 
carriers  (regionally  and  nationally);  2)  within  a  file  category  -  comparison 
with  previous  year's  data  for  a  carrier  (and  with  comparable  regional  and 
national  average  comparison  data);  and  3)  for  each  carrier,  comparison  among 
the  four  BMAD  files  where  appropriate.    These  types  of  analytical  reports  have 
two  applications:    1)  use  by  the  analyst  in  determining  any  major  problems  with 
BMAD  files  and  2)  use  by  researchers  intending  to  use  BMAD  data.    For  the  first 
application,  the  reports  can  highlight  areas  of  consistency  either  among 
carrier,  or  for  a  given  carrier  compared  to  previous  years's  data.  For 
researchers,  the  reports  indicate  significance  of  carrier  reporting  inconsis- 
tencies and  errors.    This  application  is  similar  to  that  of  the  existing  BMAD 
Extract  system,  with  the  major  difference  being  that  these  reports  describe 
carrier  specific  data  rather  than  the  primarily  aggregate  data  in  this  Extract 
System. 

A  detailed  description  of  the  proposed  edits  and  analytical  reports 
has  been  provided  in  our  previously  submitted  reports.    We  understand  that  BDMS 
has  increased  the  extent  of  automated  review  of  BMAD  data,  beginning  with  the 
CY1986  submission.    We  concur  with  these  steps  and  recommend  that  any 
additional  automation  required  be  implemented  as  soon  as  possible. 


5.  A  comprehensive  data' management  system  should  be  implemented  related 

to  the  submission  and  maintenance  of  original  BMAD  data. 

As  described  in  our  previously  submitted  reports,  many  of  the 
original  1984  BMAD  tapes  and  some  of  the  original  1985  BMAD  tapes  are 
reportedly  no  longer  available.    We  recommend  that  an  improved  data  management 
procedure  be  implemented  whereby  a  copy  is ^immediately  made  of  the  original 
tape  submitted  by  the  carrier,  followed  by  secure  storage  of  the  original. 
Currently,  BDMS  converts  incoming  carrier  tapes  to  its  internal  working  format 
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and  subsequently  uses  these  converted  files  for  most,  if  not  all,  of  its 
processing.    Since  the  conversion  process  may  have  bugs,  the  internal  files 
should  not  be  considered  as  adequate  backup  for  the  originals.    If  it  is  deemed 
too  cumbersome  for  HCFA  to  maintain  two  copies  of  the  carrier's  files,  then  the 
carrier  should  be  instructed  to  maintain  a  copy  for  a  specific  time  period, 
such  as  two  years. 


6.  The  process  used  to  generate  the  BMAD  files  should  be  simplified. 

The  current  BMAD  system  does  not  lend  itself  to  an  audit  trail.  Some 
data,  such  as  in  the  Procedure  File,  are  aggregated  and/or  derived  using  the 
entire  claims  history  file,  whereas  the  Provider  and  Beneficiary  files  are 
samples.    A  better  arrangement  might  be  a  two-stage  process.    In  the  first 
stage  all  relevant  variables  would  be  extracted  from  the  carrier's  claims 
history  file,  and  various  standard  reports  would  be  issued  based  on  the 
resultant  database.    This  extract  would  include  only  basic  variables  and,  for 
this  reason,  it  should  be  easier  for  the  carriers  to  accurately  translate  the 
instructions  for  their  particular  system.    The  second  stage  would  involve 
detailed  instructions  for  the  aggregation,  derivation,  sampling,  etc.  of  the 
stage  1  information  -  the  generation  of  BMAD  files.    The  global  reports  issued 
as  part  of  the  first  stage  should  make  it  easier  to  isolate  problems.    In  fact, 
if  the  first  stage  output  format  is  well  defined,  generation  of  the  second 
stage  could  be  completed  by  HCFA  (and  be  provided  to  the  carriers).  We 
understand  that  the  new  CABLE  system  includes  a  process  similar  to  this,  in 
that  BDMS  will  be  generating  the  BMAD  file  using  payment  data  submitted 
directly  by  the  carrier. 


7.  The  BMAD  conference  with  carrier  representatives,  to  discuss  BMAD 

reporting  issues,  should  be  continued. 

This  exchange  of  ideas  and  problems  has  been  useful  for  both  HCFA 
staff  (including  ORD,  BERC,  BPO,  OLP,  and  others  in  addition  to  BDMS)  and 
carrier  personnel.    For  example,  it  has  been  through  this  process  that 
significantly  more  information  was  obtained  concerning  such  issues  as: 

o     adjustments  -  both  internal  processing  by  carriers  and 
difficulties  in  accurate  reporting  in  BMAD; 

o     different  criteria  used  for  assignment  of  provider  ID 
numbers;  and 

o     reasons  for  different  reporting  of  allowed  charges  (i.e., 
application  of  beneficiary  specific  limitations). 

We  recommend  that  such  conferences  be  held  on  at  least  an  annual 
basis,  particularly  prior  to  the  full  CABLE  conversion. 
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